The Performance Cost of Virtual Machines on Big Data Problems in Compute Clusters

To facilitate better management of large data-intensive compute clusters, many cluster owners and providers of cloud computing environments are looking at virtualization technology as a potential solution. However, virtual machines exhibit performance degradation when compared with physical machines since a virtual machine is unable to execute privileged instructions without first going through the virtualization software. The purpose of our research is to construct a cloud computing environment and assess the performance cost of large-scale cluster computing on large data sets using virtual machines as opposed to physical machines. We developed and tested three different workloads: a synthetic disk-dominant application, a CPU-intensive real-world scientific modeling application, and a real-world ground density modelling application that exhibits a hybrid of CPU and disk operations. These workloads were then evaluated on Denison’s own compute cluster running both physical and virtual machines to assess the performance penalties that may be incurred by using a cluster of virtual machines.